Skip to content

fix(cli): spec-compliant terminal image rendering + Windows clipboard… - #974

Closed
smarpitm wants to merge 2 commits into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering
Closed

fix(cli): spec-compliant terminal image rendering + Windows clipboard…#974
smarpitm wants to merge 2 commits into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering

Conversation

@smarpitm

@smarpitm smarpitm commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown

What

Fixes terminal inline image rendering so attached photos display correctly in kitty-capable terminals, and hardens Windows clipboard image paste.

Why

Photos attached via /image or Ctrl+V could render "distorted":

  1. Kitty graphics protocol spec violations — the final chunk omitted the required m=0 terminator and every chunk repeated a=T/full control data, so terminals treated each chunk as a new image (fragmented display). The format id was also hardcoded to f=100 (PNG) even for the JPEG payloads the compression step produces.
  2. iTerm2 size param sent the base64 length instead of the decoded byte length (minus padding).
  3. Detection missed WezTerm / Ghostty / Warp / Konsole, which all support kitty graphics.
  4. Windows paste failed silently on machines without powershell.exe (no pwsh fallback).

Changes

  • cli/src/utils/terminal-images.ts — spec-compliant kitty chunking (m=0 on the last chunk, control data only on the first), format id derived from the real media type, iTerm2 size = decoded bytes, detection for WezTerm/Ghostty/Warp/Konsole (case-insensitive).
  • cli/src/utils/clipboard-image.ts — pwsh fallback when powershell.exe is missing, normalized string output.
  • cli/src/components/blocks/image-block.tsx, image-card.tsx — pass mediaType to the renderer.
  • New tests: terminal-images.test.ts (sequence conformance against the kitty spec), image-pipeline-integrity.test.ts (verifies image processing never corrupts payloads).
  • test/setup-scm-loader.ts — no-op stub so bun test in the cli package works in the public mirror (referenced by cli/bunfig.toml but missing from the public repo).

Validation

  • bun run typecheck (cli) clean.
  • 25 new tests + 152 related tests pass.

Note

DeepSeek V4 Flash (the default model) is multimodal: false and reads images via the server-side describe pipeline — for best results with photos, use MiniMax M3 / GPT-5.6 Luna / MiMo 2.5.

@codebuff-team

codebuff-team commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Good catches on terminal-images.ts: the missing m=0 terminator on the final chunk, repeating full control data (a=T, f=, c=, r=) on every chunk instead of just the first, and the iTerm2 size param using base64 length instead of decoded byte length — all three are genuine, well-described spec violations and worth porting.

But getKittyFormat() (terminal-images.ts) invents format ids that don't exist in the actual Kitty graphics protocol. Per the spec (https://sw.kovidgoyal.net/kitty/graphics-protocol/), f only takes 24 (RGB), 32 (RGBA), or 100 (PNG) — there is no 101/102/103/104 for PNG-with-alpha/JPEG/WebP/GIF. Since image-handler's compression path produces raw JPEG bytes (per your own new image-pipeline-integrity.test.ts), sending those bytes with f=102 to a real kitty/WezTerm/Ghostty terminal will not decode — the terminal has no such format and will either error or silently drop the image. This reintroduces exactly the 'distorted image' bug the PR is meant to fix, just for the compressed-JPEG code path instead of the chunking path. The tests you added only assert that the escape sequence contains f=102, not that any real terminal accepts it, so they don't catch this.

Before this can be ported: either convert compressed images to PNG before the kitty path (simplest, since 100 is the only guaranteed-compatible non-raw format), or drop the fabricated format ids and stick to 100/24/32 as the spec actually defines, deriving alpha/no-alpha via raw pixel mode if you want to avoid re-encoding.

The terminal-detection additions (WezTerm/Ghostty/Warp/Konsole) and the Windows pwsh fallback in clipboard-image.ts look solid independently and could be split into a smaller, separately reviewable PR.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written labels Aug 20, 2026
@smarpitm @codebuff-team
fix(cli): spec-compliant kitty format ids + PNG conversion for non-PN…
b30b35b 
…G payloads

getKittyFormat() now always returns 100 (PNG) — the only format id that
all kitty-protocol terminals guarantee. Non-PNG payloads (JPEG, WebP, GIF)
are converted to PNG via Jimp before transmission, so the bytes always
match f=100. This prevents terminals from silently dropping images due to
fabricated format ids (101-104) that don't exist in the kitty spec.

renderInlineImage() is now async to support the Jimp conversion step.
image-block.tsx and image-card.tsx updated to handle async with
useEffect + state.

New tests validate that the escape sequence uses only spec-compliant
format ids (f=24|32|100) across all media types, and that no fabricated
ids (101-104) appear in any chunk.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
@smarpitm smarpitm closed this Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@smarpitm @codebuff-team